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TITLE OF THE INVENTION 

METHOD AND APPARATUS FOR PROCESSING ATVEF DATA TO CONTROL THE 

DISPLAY OF TEXT AND IMAGES 

BACKGROUND OF THE INVENTION 
FIELD OF THE INVENTION 

[0001] The present invention generally relates to the use and processing of interactive 
television data for dehvering enhanced television programming in a CATV environment. 

BACKGROUND INFORMATION 

[0002] The Advanced Television Enhancement Forum (ATVEF) was formed in 1 997 by a 
consortium of 14 leading companies in the television and computing industries. This group 
developed a public, worldwide specification for creating and delivering interactive TV (ITV) 
content In 1999, the ATVEF Specification v 1.1, r26 was fmalized and published. The ATVEF 
Specification serves as a standard for creating enhanced, interactive television content and 
delivering that content to a range of television, set-top, and PC-based receivers. The ATVEF 
Specification uses existing Internet technologies to deliver enhanced TV programming over 
both analog and digital video systems using terrestrial, cable, satellite and Internet networks. 
The ATVEF Specification can be used in both one-way broadcast and two-way video systems, 
and is designed to be compatible with all international standards for both analog and digital 
video systems. 

[0003] Television enhancements are comprised of three related data sources: 
announcements (deUvered via SAP), content (deUvered via UHTTP), and triggers (deUvered via 
the trigger protocol over UDP), SAP (Session Announcement Protocol) is a protocol used for 
session announcements. UHTTP (Unidirectional Hypertext Transfer Protocol) is a simple, 
robust, one-way resource transfer protocol that is designed to efficiently deliver resource data in 
a one-way broadcast-only environment. UDP (User Datagram Protocol) is an Internet Standard 
transport layer connection-less protocol which adds a level of reUability and multiplexing to IP. 
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IP is one of the communication languages used by computers connected to the Internet. This 
resource fransfer protocol is appropriate for Internet Protocol (IP) multicast over a television 
vertical blanking interval (VBI), IP multicast carried over MPEG, or other unidirectional 
transport systems. MPEG (Moving Picture Experts Group) is the name of a family of standards 
used for coding audio-visual information (e.g., movies, video, music) in a digital compressed 
format. The major advantage of MPEG compared to other video and audio coding formats is 
that MPEG files are much smaller for the same quality. This is because MPEG uses very 
sophisticated compression techniques. 

[0004] In 1953, the NTSC (National Television Standards Committee) developed a set of 
standard protocols for television (TV) broadcast transmission and reception for use in the 
United States. An NTSC TV image has 525 horizontal lines per frame (complete screen 
image). These lines are scanned fi-om left to right, and from top to bottom. Every other hne is 
skipped. Thus, it takes two screen scans to complete a frame, one scan for the odd-numbered 
horizontal Unes, and another scan for the even-numbered lines. Each half-frame screen scan 
takes approximately 1/60 of a second. A complete frame is scanned every 1/30 second. Each 
time the electron gun in the television's cathode ray tube finishes scanning a half-frame, it must 
return to the upper left hand comer of the television screen to prepare for the next half-frame. 
This takes a significant amount of time, so each pass of the electron gun must be synchronized 
with the incoming signal. This is done by adding a set of unused Unes of data (exactly 21 lines) 
to the end of each screen scan, giving the electron gun time to return to its startmg position. 
These 21 extra lines make up the VBI. The VBI of a television signal is a non-viewable portion 
of the television signal that can be used to provide point-to-multipoint IP data services and 
reUeve congestion and traffic in the traditional Internet access networks. IP datagrams may be 
transmitted using the VBI of a television signal. 

[0005] Only the first nine lines of the VBI are actually required to reposition the cathode 
ray. This leaves twelve more lines (10 through 21) that can be used to broadcast data. In the 
United States, closed captioning data is broadcast on VBI hne 21. The transmission of 
multicast-IP using the North American Basic Teletext Standard (NABTS) is a recognized and 
industry-supported method of transporting data on the VBI. NABTS has traditionally been 
used on 525 hne television systems such as NTSC. 
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[0006] The ATVEF Specification defines how ATVEF content is displayed and how the 
receiver is notified of new content. The ATVEF Specification also defines how content is 
delivered. Since a television or set-top terminal does not necessarily have a connection out to 
the Internet, the ATVEF Specification describes two distinct models for delivering content. 
These two content delivery models are commonly referred to as transports, and the two 
transports defmed by ATVEF are referred to as transport type A and transport type B. 
[0007] Transport type A is defmed for ATVEF receivers that maintain a connection 
(commonly called a back-channel or return path) to the Internet. Transport A is for delivery of 
triggers by the forward path and the puUing of data by a (required) return path. Generally, this 
network connection is provided by a dial-up modem, but may be any type of bi-directional 
access channel. Transport type A is a method for dehvering only triggers without additional 
content. Since there is no content delivered with Transport type A, all data must be obtained 
over the back-channel, using URLs passed with the triggers as a pointer to the content. ATVEF 
triggers are broadcast in line 21 of the VBI. 

[0008] Transport type B provides for delivery of both ATVEF triggers and its associated 
content via the broadcast network. In tiiis model, the broadcaster pushes content to a receiver, 
which will store it in the event that the user chooses to view it. Transport B uses 
annoxmcements sent over the network to associate triggers with content streams. An 
announcement describes a content stream, and may include information regarding bandwidth, 
storage requirements, and language (enhancements may be delivered in multiple languages). 
Transport Type B uses VBI Hnes 10 through 20 to carry interactive television information, such 
as IP datagrams. 

[0009] Thus, with the advent of ATVEF and digital cable TV, a new era in TV viewing 
experience is emerging in which video complementary data services are available to the TV 
viewer. There are many potential methods for experiencing these data services. One such way 
is to use a second display screen on an auxiliary display device. One example of an auxihary 
display device is a webpad, which is a relatively small remote wireless device. 
[0010] Auxiliary display devices are Internet compatible appliances that process and 
display ATVEF data. The auxiliary display devices are often required to display broadcast 
television channel video images in a designated area of the screen on the auxiliary display 
devices. The broadcast television channel video image is normally referred to for obtaming 
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image map data, which defines how ATVEF data and broadcast television channel video 
images are to be displayed. Under certain circumstances, some auxiliary display devices are 
unable to access to the video image. Thus, an area of the screen designated for the broadcast 
television channel video image may not be available for use. 

5 SUMMARY OF THE INVENTION 

[0011] In today's cable television market, subscribers are provided with the opportunity to 
access enhanced video services. Typically, these enhancements include some type of television 
content metadata and/or IP/World Wide Web information transmitted by a video service 
5 provider to a set-top box (STB). The viewer is then allowed to select this metadata (enhanced 
II 0 information) using an auxiliary display device. The same auxiliary display device is used to 
14 control the selection of programming viewed on a television. The content metadata is displayed 
% on a display of the auxiliary display device. The present invention enables an alternative 
SI predefined image (e.g., an advertisement or an image selected by a user of the auxiliary display 
device) to be displayed in an area of the display of the auxiliary display device reserved for the 
k| 5 broadcast television channel video image. The present invention establishes more control over 
|| what is displayed on the auxiliary display device. The present invention may be implemented 
ly Internet apphances, cable/satellite television access control protocols, or any auxiliary 
display devices equipped with a web browser. 

[0012] In a preferred embodiment of the present invention, television content metadata is 
20 processed in a communications system that includes a set-top box (STB) and an auxiUary 
display device. The auxiliary display device includes a memory, a display, a processor and a 
metadata processing application. The memory stores a predefmed image. The STB extracts 
television content metadata from a transport stream received by the STB. The extracted 
metadata defines at least one of text and images. The extracted metadata is transmitted firom the 
25 STB to the auxiliary display device. The extracted metadata is processed in the STB using the 
metadata processing application running on the processor of the auxiliary display device. The 
predefined image stored in the memory of the auxiliary display device, and the at least one of 
text and images defined by the extracted metadata are adjacently displayed on the display of the 
auxiliary display device. 
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[0013] The memory of the auxiliary display device may store a plurality of predefined 
images. The metadata processing application of the auxiliary display device may change the 
displayed predefined image on a periodic basis. The extracted metadata may mclude a uniform 
resource identifier (URI). The metadata processing application of the auxiliary display device 
may change the displayed predefmed image each time the auxiliary display device receives a 
URI fi:om the STB. The extracted metadata may further include a uniform resource identifier 
(URI) that specifies a particular area on the display of the auxiliary display device for a 
broadcast television channel video image to be presented. The metadata processing appUcation 
of the auxUiary display device may replace the URI with another URI stored in the memory of 
the auxiUary display device. The extracted metadata may specify a format for displaying at 
least one of images and text on the display of the auxiliary display device. The metadata 
^ processing apphcation of the auxiliary display device may change the format specified by the 
J extracted metadata. The predefined image may be an advertisement. The extracted metadata 

may be advanced television enhancement forum (ATVEF) data. The transport stream may 
j|5 inchide a pkralityofvertical blanking interval (VBI) lines. The metadata may be extracted 
j| firom at least one of the VBI lines. The transport stream may be a Moving Picture Experts 

13 Group (MPEG) transport stream. The extracted metadata may be stored and then be 

13 

transmitted to the auxiliary display device at a later tune in response to playing back the stored 
metadata. 

20 BRIEF DESCRIPTION OF THE DRAWINGS 

[0014] The following detailed description of preferred embodiments of the present 
invention would be better understood when read in conjunction with the appended drawings. 
For the purpose of illustrating the present invention, there are shown m the drawings 
embodiments which are presently preferred. However, the present invention is not limited to 
25 the precise arrangements and instrumentalities shown. In the drawings: 

[0015] FIG. lA is a block diagram of a communications system operating in accordance 
with the present invention; 

[0016] FIG. IB shows how display space is allocated for presenting text and/or images, and 
an alternate predefined unage, in the communications system of FIG. 1 A; 
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[0017] FIG. 2 shows a packet configuration for the forwarding of vertical blanking interval 
(VBI) line 21 ATVEF data in accordance with the present invention; 
[0018] FIG. 3 shows an IP datagram reassembled firom NABTS packets transported in the 
VBI in accordance with the present invention; 

[0019] FIG. 4 shows m IP datagram reassembled from MPEG transport packets in 
accordance with the present invention; 

[0020] FIG. 5 is a data flow diagram of an ATVEF VBI hne 21 process hnplemented m 
accordance with the present invention; 

[0021] FIG. 6 is a data flow diagram of an ATVEF VBI hnes 10 through 21 process 
implemented hi accordance with the present invention; 

[0022] FIG. 7 is a data flow diagram of an ATVEF over MPEG process wherein ATVEF 
content is displayed on the display of an auxiliary display device in accordance with the present 
invention; 

[0023] FIG. 8 is a data flow diagram of an ATVEF over MPEG process wherein ATVEF 
content is displayed on a television screen in accordance with the present invention; 
[0024] FIG. 9 is a data flow diagram of an IP over MPEG process (data service) in 
accordance with an alternate embodiment of the present invention; and 
[0025] FIG 10 is a high-level functional flowchart including steps implemented by the 
communications system of FIGs. 1 A and IB. 

DETAILED DESCRIPTION OF THE INVENTION 

[0026] The present mvention provides a video entertainment service subscriber with the 
capability of directing enhanced content (television content metadata) to a device other than the 
primary receiver (e.g., a television). 

[0027] FIG. lA shows a communications system 100 including an auxiliary display device 
1 10 that communicates with an STB 120 via path 130. The auxiliary display device 1 10 
includes a processor 1 12 which executes a television content metadata (e.g., ATVEF) 
processmg appUcation 1 14. The auxihary display device 1 10 also mcludes a display 1 16. STB 
120 is used by a subscriber of video services from service provider 140. Content provided by 
service provider 140 is presented on television 150 via STB 120 and path 180. Path 130 may be 
a wired or wireless connection. The wireless connection utilizes an external port 122 of STB 
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120, such as a universal serial bus (USB), Ethernet, or IEEE 1394 port equipped with a dongle 
160 (e.g., a wireless local area network (WLAN)). A dongle is a device used for ensuring that 
only authorized users can copy or use specific software applications. Dongle 160 is used to 
support wireless connectivity between STB 120 and auxiliary display device 1 10. The auxiliary 
display device 1 10 may access Internet 170 via STB 120. The auxiliary display device 110 may 
also be connected directly to a high-speed cable modem, digital subscriber (DSL) modem or 
any other high-speed Internet connection device to access the Internet 170. The wireless 
connection could be either HomeRF® or IEEE 802. 1 1. A more traditional wired connection 
would simply include a cable or wire between STB 120 and auxiliary display device 110, again, 
using a USB, Ethernet, or IEEE 1394 port. 

[0028] The STB 120 includes an ATVEF delivery function 124 which receives enhanced 
content/metadata from service provider 140 via a transport stream 145. ATVEF dehvery 
function 124 is configured to perform the following fimctions: 

[0029] (1) Pass-through all ATVEF data to the auxiliary display device 1 10 for processmg 
(analog ATVEF, VBI line 21 data, and VBI lines 10 through 21 data) (see FIGs. 5 and 6); 
[0030] (2) Pass-through content and triggers to the auxiliary display device 1 10 for 
processing (ATVEF over MPEG) (see FIG. 7); 

[0031] (3) Selectively process certain ATVEF and television content metadata locally for 
display on the television 150, before passing through the remaining data to the auxiliary display 
device 1 10 (see FIG. 8); and 

[0032] (4) For the processing of a primary data service, displaying optional video and audio 
content, when present, on television 1 50 or the auxiUary display device 1 10 such that all data is 
routed to the auxiliary display device 1 10 for processing (see FIG. 9). 

[0033] FIG. IB shows how television content metadata extracted from transport stream 145 
is used to configure text and/or images on a screen 152 of television 150 and the display 1 16 of 
auxiliary display device 1 10. The television content metadata includes a uniform resource 
identifier (URT) that requires that a broadcast television channel video image be presented in a 
designated location. For example, the television 150, which has access to the video image, 
displays the broadcast television channel video unage at location 156 on screen 152. At the 
same time, text and/or otiier images may be displayed at location 154 on screen 152 of 
television 150. The auxihary display device 1 10, which does not have access to the video 
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image, displays a predefined image at location 128 on display 1 16 of the auxiliary display 
device 1 10. The same text and/or images shown at location 154 on screen 152 of television 150 
may also be shown at location 126 on display 116 of the auxiliary display device 1 10. The 
predefined image may either be obtained fi-om a memory 118 within the auxiliary display 
device 110 that was loaded during the manufacture of the auxiliary display device 1 10, or 
accessed from a remote site, such as Internet 170. The display of the predefined image can be 
dependent on an algorithm running on processor 1 12 within the auxiliary display device 1 10. 
For example, the predefined image may change each tune a new "tv:" URL is received by the 
auxiliary display device 110 firom STB 120. In another example, the memory may store a 
plurality of predetermined images. The predefined image displayed on the auxiUary display 
device 1 10 may be changed on a periodic basis (e.g., every 30 seconds) using a timmg 
mechanism (not shown). Any display area defined hyperlmks that are specified by the 
extracted metadata are remapped to either the predefined image, or some other presentable 
format. 

[0034] The auxiliary display device 1 1 0 presents the predefined image in an area of display 
116 designated by the metadata as being reserved for a broadcast television channel video 
image. The predefined image may be an advertisement. The television 150 presents a 
broadcast television channel video image adjacent to the text and/or unages defmed by the 
metadata. The URI may be a uniform resource locator (URL) used to launch a web page that 
provides display location data that specifies how the text and/or images defined by the 
metadata, and the predefined image, are to be presented on the display 116 of the auxiliary 
display device 1 10. The auxihary display device 1 10 may convert the URI into a different URI, 
using conventional correlation techniques. 

[0035] A toggle function can be incorporated into the STB 120 and/or auxiliary display 
device 1 10 to allow a user to select whether metadata extracted fi-om transport stream 145 
should be processed by television 150, assummg it has an ATVEF or other metadata processing 
apphcation running within, or by the auxihary display device 1 10. Alternatively, the extracted 
metadata can be stored by a personal video recorder (PVR), and transmitted fi-om the STB 120 
to the auxihary display device 1 10 at a later time when the stored metadata is played back by 
the PVR. The PVR can be mcorporated into STB 120 or it can be configured as an independent 
device that communicates with STB 120. The metadata can also be stored in the STB 120, in 
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another auxiliary device (e.g., an external hard drive), or in a remote server. An authoring 
mechanism and application programming interface (API) set may be incorporated into system 
100 to enable a content creator to determine what content is to be viewed on television 150 
Mid/orthe auxiliary display device 110. 

[0036] A key functionality of the ATVEF Delivery Function 1 24 of STB 1 20 is the 
packetization of raw ATVEF content metadata received from transport stream 145 on VBI line 
21 . The ATVEF content metadata is carried on VBI hne 21 using a Text-2 (T-2) service. The 
ATVEF content metadata is configured to include at least one uniform resource locator (URL). 
The ATVEF content metadata is extracted from the VBI via the ATVEF Delivery Function 124. 
Once extracted, the ATVEF Dehvery Function 124 will verify that the metadata is ATVEF. 
[0037] The ATVEF metadata defmes text and/or images displayed on the auxiliary display 
device 1 10 and/or the television 150. The images may mclude static video images, fijll 30 
frames per second MPEG-2 video, or the hke. The ATVEF metadata may also define audio, 
animation (e.g.. Flash™, Shockwave®, or the hke), video clip or any other type of media that 
can be presented from Internet content. 

[0038] FIG. 2 shows how VBIhne 2 1 metadata is processed by the ATVEF Delivery 
Function 124 of STB 120 by constructing a packet to forward the VBI hne 21 data. If the 
metadata is ATVEF, then the ATVEF Delivery Function 124 encapsulates the metadata into a 
SAP packet. The ATVEF Dehvery Function 1 24 then encapsulates the SAP packet into a 
UDP/IP packet that is assigned a multicast IP and a UDP destination port. The ATVEF 
Dehvery Function then transmits a UDP/IP datagram via port 122 of STB 120. 
[0039] FIG. 3 shows how IP ATVEF data received on VBI Unes 10 through 20 is processed 
by the ATVEF Dehvery Function 124 of STB 120 by packetizing IP ATVEF data in the form 
of NABTS packets. ATVEF data that is carried in IP over VBI is extracted from VBI lines 10 
through 20 via the ATVEF Dehvery Function 124, reconstructed mto serial line Internet 
protocol (SLIP) packets, encapsulated mto UDP/IP packets and passed to the port 122 of STB 
120. 

[0040] FIG. 4 shows how IP ATVEF data received on the MPEG transport stream is 
processed by the ATVEF Dehvery Function 124 by packetizmg IP ATVEF in the form of 
MPEG Transport packets. ATVEF data that is carried in IP over MPEG is extracted from the 
MPEG transport stream via the ATVEF Dehvery Function 124, reconstructed mto digital video 
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broadcasting multi-protocol encapsulation (DVB-MPE) packets, encapsulated into UDP/IP 
packets and passed to tiie port 122 of STB 120. 

[0041] FIG. 5 shows an embodiment of the present invention wherein analog ATVEF data 
including at least one uniform resource identifier (IM) is extracted fi-om VBI line 21 in the 
STB 120 and forwarded for processing in the auxiUary display device 1 10. A URI identifies 
any metadata resource. The URI may be a uniform resource locator (URL) which locates a 
metadata resource of the World Wide Web. In this example, the ATVEF processing function in 
the STB 120 is configured for pass-through capability only, and the STB 120 does not process 
any ATVEF data locally. Video and audio for this service is displayed to the user's television 
150. For analog ATVEF, the URI contamed in VBI Une 21 is sent to the auxiliary display 
device 1 1 0 and presented on display 1 1 6 to the user. Here, a return channel is used by the 
auxiliary display device 1 10 to access actual content. 

[0042] FIG. 6 shows another embodiment of the present invention wherein VBI Unes 1 0 
through 21 are used in an analog video service to convey ATVEF announcements, content and 
triggers. The analog video and audio are routed to the user's television 150, and ATVEF data is 
routed to the auxiliary display device 110. Optionally, ATVEF data may be processed 
internally on the STB 120 for presentation on the user's television 150. In this example, 
however, the display 1 16 of the auxiliary display device 1 10 is used for the presentation of 
enhanced content. 

[0043] FIG. 7 shows yet another embodiment of the present invention wherein the user has 
tuned to an enhanced digital video service that contams both content and triggers. In tihis 
embodiment, the ATVEF Delivery function 124 in the STB 120 is configured differently than 
ui the analog ATVEF case. No separate return channel is required to retrieve content, smce all 
content is present in the video service multiplex. Once the service is tuned, an icon is displayed 
on both the television 1 50 and the display 1 1 6 of the auxiliary display device 1 1 0 to inform the 
user that this is an enhanced service. Using a keyboard (not shown) on the auxiliary display 
device 1 1 0, the user selects the icon on the display 1 1 6 of the auxiliary display device 1 1 0, 
causing all content and trigger information to be sent directly to the auxiliary display device 110 
for processing and display. Normal video and audio continues to be displayed to the television 
150. 
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[0044] FIG. 8 shows yet another embodiment of the present invention wherein the user can 
select the icon displayed on the television 1 50, in which case all content and triggers are 
processed internally by the ATVEF dehvery function 124 in the STB 120 for display on the 
user' s television 1 50. 

5 [0045] FIG. 9 shows yet another embodiment of the present invention wherein a data 
service may optionally contain video and audio. If present, the STB 120 routes the video and 
audio to the user's television 1 50 by default. All metadata is routed directly to the auxiliary 
display device 1 10. Data may optionaUy be selectively routed to the television 150, at the 
discretion of the user. 

||0 [0046] FIG. 10 shows how the preferred embodiment of the present invention is 

implemented in communications system 100. In step 1005, the STB 120 extracts television 
content metadata from transport stream 145 received by the STB 120. The extracted metadata 
defmes at least one of text and images. In step 1010, the extracted metadata is transmitted from 
the STB 120 to the auxiUary display device 110. Alternatively, the extracted metadata is stored, 
C|5 and then transmitted to the auxiUary display device 1 10 at a later time when the stored metadata 
is played back. In step 101 5, the metadata processing application 1 14 runnmg on the processor 
1 12 of the auxiliary display device 1 10 is used to process the extracted metadata received from 
f ll STB 120. In step 1020, the auxiliary display device 110 retrieves a predefmed image from 

memory 118. In step 1025, the predefined image 128 is displayed adjacent to the at least one of 
20 text and images 126 defmed by the extracted metadata on display 1 16 of the auxiUary display 
device 1 10. Alternatively, the predefmed image may be accessed from a remote site or other 
source, such as from Internet 170. In step 1 030, a determination is made as to whether an 
optional periodic timing function is to be initiated by tiie metadata processing appUcation 1 14, 
whereby the memory 118 stores a plurality of predefmed images and the displayed predefined 
25 image is changed on a periodic basis (e.g., every 30 seconds). If the optional periodic timing 
function is implemented, and a predetermined time period elapses (step 1035), the auxiUary 
display device 1 10 retiieves a different predefined image from memory 1 18 as instructed by tiie 
metadata processing appUcation 114 (step 1020). Furthermore, if the auxiUary display device 
1 10 receives new metadata from STB 120 (step 1040), the auxiUary display device 1 10 will 
3 0 retrieve a different predefined image from memory 1 1 8 (step 1 020). 
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[0047] The present invention may be implemented with any combination of hardware and 
software. If implemented as a computer-implemented apparatus, the present invention is 
implemented using means for perfonning all of the steps and functions described above. 
[0048] The present invention can be included in an article of manufacture (e.g., one or more 
computer program products) having, for instance, computer useable media. The media has 
embodied therein, for instance, computer readable program code means for providing and 
facilitating the mechanisms of the present invention. The article of manufacture can be 
included as part of a computer system or sold separately. 

[0049] It will be appreciated by those skilled in the art that changes could be made to the 
embodiments described above without departing from the broad inventive concept thereof It is 
understood, therefore, that this invention is not limited to the particular embodiments disclosed, 
but it is intended to cover modifications within the spirit and scope of the present invention as 
defined by the appended claims. 
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